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ABSTRACT 



A multi-case study was used in the self-study of three 
programs within an academic department of a mid-sized Southern university. 
Multi-case methodology as a form of self-study encourages a process of 
self-renewal and programmatic change as it defines an active stakeholder 
role. The participants in. the three case studies were university faculty and 
administrators, current graduate students, graduates of the secondary teacher 
preparation program, and members of program advisory groups. Data collection 
techniques included surveys, focus groups, and the multi-case method itself. 

A grounded theory approach was used to analyze the data for each study. 

During the study, the frame of self-study emerged within the research 
process, resulting in a significant shift in the epistemological paradigm and 
attitudes of the faculty related to scholarly activities. This shift has 
further resulted in an evolutionary process of ongoing self -renewal within 
the case study analysis. The academic unit's faculty has substantially 
increased its commitment to the inquiry process at several levels. The 
faculty has become a living artifact in, and as a result of, the method. The 
evolutionary process of self-study, self -renewal , and mindful inquiry 
provided the opportunity for the faculty to engage in critical 
self-examination. (Contains 16 references.) (SLD) 
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Methodological Considerations for an Evolving Model of 

Institutional Research 



Wisdom not steeped in methods is bondage. 
Wisdom steeped in method is freedom. 

Method not steeped in wisdom is bondage. 

Method steeped in wisdom is freedom. 

Tsongkapa, The Principal Teachings of Buddhism 



The threads of self-study, self-renewal, and mindful 
inquiry are woven together within multicase methodology to 
provide an evolving picture of a department in transition. 
The emerging patterns of coherence, engagement, and 
authenticity have contributed to the developing scholar- 
practitioner culture. Multicase methodology is an holistic, 
naturalistic, interpretive approach to institutional 
research which also encompasses a treinsf ormational process. 
Feedback loops allow revxew and redesign of existing 
programs, while feed-forward loops inform the change process 
and connect faculty to a continuous process of self -renewal. 
Review of the Literature 

Any social arreuigement that remains vitally social, or 
vitally shared, is educative to those who participate 
it* John Dewey, Democracy and Education 

In the fall of 1993, a distinguished group of higher 

education and industry leaders, the Wingspread Group, issued 

a 160-page report entitled An Americein Imperative; Higher 



Expectations for Higher Education . The group opened its 
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report wxth a dxsconcertxng statement ahoxit the 
effectiveness of higher education in America: "A disturbing 

and dangerous mismatch exists between what Americ 2 m society 
needs of higher education and what it is receiving" 

(Seymour, 1995, p. x) . Over the past decade much has been 
written about institutional effectiveness and the need for 
new directions in academic programs. Seymour (1995) suggests 
that "the chasm between what we do in higher education 
what society needs must be bridged" (p. xi) . In an 
environment where external demands abound, serious internal 
excunination is essential (Wergin, 1987) . 

Instxtutxons of hxgher education must change the way 
they look at themselves. Current literature substantiates 
the need for a greater emphasis on internal monitoring « nd 
”®xa®ination that is closely linked to orgeuii rational 
improvement and self-renewal (Wergin & Braskamp, 1987; 

Haworth eind Conrad, 1997; Kells, 1995). 

(1987) suggests that the process of rigorous 
®®lf ”®X3mination is indeed the process of renewal itself. At 
its best, self-study becomes "a process of critical inquiry 
that monitors the health of the [institution], while at «-h o 
same time legitimating and providing a forum for critical 
discourse and action by those who work in the [institution]" 



Methodological Considerations 4 



(Sirotnik, 1987, p. 42). As the self-study process becomes 
deeply ingrained in the culture of the institution, self- 
renewal is imminent and new patterns of development begin to 
evolve within the culture of the organization. 

inquiry, when used as process for studying 
institutional effectiveness is centered on learning; its 
outcome is the production of critical knowledge. According 
Sirotnik, "critical knowledge is the consciousness 
raising and enlightenment gained through reflection on, 
critique of, existing knowledge through the dialectical 
process embedded in the questions we ask" (1987, p. 51). 

The ultimate goal of self-study through critical 
inquizy is increasing the effectiveness of the organization 
while ensuring the growth and development of those involved 
in the process (Seymour, 1995) . When critical inquiry becomes 
an integral part of the institution's way of doing things, 
the practitioner learns to function as a researcher a 

scholar (Schon, 1983 ) . The process itself becomes a source 
of ongoing self-renewal for the individuals as well as the 
organization. 

S©lf~study through critical inquiry provides colleges 
and universities a way of looking at themselves for the 
purposes of growth and improvement. Ultimately, the self- 
study process must involve everyone. It must become a part 
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of the institution's conscious and deliberate pursuit of 
excellence (Wergin, 1987). 

Developing and sustaining high-quality progreuns 
requires those who have a stake in academic programs to 
engage in an on-going process of self-renewal through 
organizational learning. "Where we have traditionally 
acknowledged the need for representation in change, it has 
tended to be a selected representation often tempered by 
politics and expediency" (Jenlink, 1996, p.75) . Quality in 
higher education demands the collective intelligence, 
engagement, and commitment of many people who mutually 
invest in their own and others' learning (Haworth & Conrad, 
1997). Wergin ( 1987 ) suggests that without genuine 
investment in the process by those who have the most to gain 

or lose, improvement- any real change- is unlikely. (Wergin, 
1987) . 

When guided by critical inquiry, the self-study 
process cultivates new, fresh ideas, from different 
perspectives and unique angles (Seymour, 1995). Through 
self-study the constructive potential for individual 
collective improvement can be realized (Sirotnik, 1987). 

^^ST^er education, self-study through critical 
inquiry may be our best hope for examining institutional 
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effectiveness while contributing to the development of 
practitioners who are committed to scholarly activity. 

Me t hodo 1 ogy 

The multicase method uses cross case comparison and 
analysis of individual case studies of academic programs as 
a means of studying both effectiveness and quality within 
the institutional research paradigm (Haworth & Conrad, 1997; 
Stake, 1995) and the development of a scholar-practitioner 
culture. It is a meta-analysis process that seeks to 
identify: 1) how patterns emerge within and across cases, 2) 
what cross patterns emerge, and 3) the meeming of the cross 
patterns that emerge. As a method of self-study it situates 
the research team in the role of self -critical examiner 
through a process of mindful inquiry (Bentz & Shapiro, 

1998). Multicase methodology as a form of self-study 
encourages a process of self-renewal and programmatic change 
while defining an active stakeholder role. 

^^^^^cipants . Stakeholder voice is a key component in 
the multicase research process (Haworth & Conrad, 1997). 
Institutional administrators, program administrators, 
students, alximni, and employers of alumni are included as 
categories of participants. The participant in the three 
case studies that form the units of analysis for this 
multicase study included: a) \iniversity faculty j»n/i 
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administrators, b) current students in the newly formed 
doctoral program, c) past graduates from the secondary 
teacher preparation program and the mid -management (K-12 
administrators) preparation program, and c) members of 
advisory groups which included superintendents and 
principals. The setting for the research was an academic 
department in a mid-size (approximately 12,400 students) 
regional university located in the south. A case study was 
developed for each of the three programs within the 
department . 

Data Collection . Data collection techniques included 
survey research, focus group interviews, (Rreuger, 1994; 
Morgan, 1988) and multicase method. Data sources for the 
three cases included a) artifacts collected from each 
faculty as they engaged in the redesign of their respective 
academic program area; b) discourse and written 
communication representing the articulation anH 
dissemination of policy and procedure designed to facilitate 
the renewal of the department in concert with the redesign 
of the academic programs; c) transcripts of audio recorded 
focus group interviews of various participemts; d) survey 
data collected from past graduates of two programs 
(secondary and mid-management) and current students of the 
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newly formed doctoral program; and e) faculty and interview 
data . 

Data Analysis . A grounded theory approach ( Straus ^ & 
Corbin, 1990) was used to analyze data collected for each 
study. As the first level of analysis, the intent of the 
case study of each program was to provide an ongoing and 
evolving study of the program. Emergent themes were 
assimilated within the research framework of patterns of 
development when moving toward a scholar-practitioner 
culture. An integrative or synthetic approach (Patton, 1990) 
to data analysis was used to create a holistic profile of 
emergent patterns. Data were sorted to address the 
procedural, structural, curricular, philosophical and 
political elements of culture as either constraining or 
developing a scholar-practitioner culture. 

The second level of analysis, multicase technique 
(Greene & David, 1981) was used to conduct a cross case 
analysis. The issues of coherence and authenticity provided 
the initial framework within which to assess the emergent 
themes provided by the three case studies. The research tezun 
examined the perspectives shared by participemts to 
determine which factors were consistent within and/or common 
across each progreun to determine coherence, while the 
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engagement of participants over time, to what extent, 
with what intent provided the indicators for authenticity. 

With the combined purpose of studying program 
effectiveness and quality, and the development of a scholar- 
practitioner culture, another level of data analysis 
exeunined the research process itself. Engagement 
authenticity were reframed within the paradigm of changing 
culture and self renewal. Level of engagement in the inquiry 
process, reflexivity as a characteristic of the process, «Ti/q 
commitment to self-critical examination provided a 
contextualization of the data. The actions performed within 
the inquiry process, as well as the actions resulting from 
engagement in the inquiry process, provided a holistic 
profile of emergent patterns within a department in 
transition. 

Results and Implications; Reflections 

Case study and cross case analysis methods applied to 
institutional effectiveness studies produce numerous 
outcomes, some of which are not typically foreseen during 
the design phase of the research. To truly appreciate the 
continuous impact the process has on an academic unit, early 
reflection at this beginning stage seems to be appropriate. 

During this study, unforeseen phenomenon occurred as 
^he frame of self-study emerged within the research process. 
For example, the self-study emergence resulted in a 




10 



Methodological Considerations lo 



significant shift in the epistemological paradigm and 
attitude of the faculty relating to scholarly activities, 
particularly research, and professional focus. This shift 
has further resulted in an evolutionary process of on-going 
self -renewal within the case study analysis. 

The full impact and longevity of this apparent shift is 
not yet known, as the academic unit is still early in the 
process. The emerging patterns of behavior are clear, 
however, encompassing a shift from the set patterns in place 
prior to the research process. The on-going, evolutionary 
nature of the self-study which has resulted from faculty 
engagement in the institutional research project, is 
wxth the scholar-practitioner model awH 
philosophy adopted by the department as a structure 
framework for its academic programs. This transformation in 
academic policy transferred into the practice of the 
department itself and influenced the shift in the 
epistemological paradigm of the faculty. It would seem that 
mindful inquiry results in scholarly practice. It may be too 
early to tell the level of scholarly activity as a result of 
the process, but clearly the focus on scholarly activities 
is evident. 

Specifically, the emerging fr5unework has resulted in 
the academic unit substantially increasing the faculty's 
commitment to the inquiry process at several levels. The 
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engagement in the research process for the institutional 
effectiveness study resulted in faculty, research assistants 
and students engaging in scholarly activities, influencing 
change in the paradigm within the academic unit . The process 
facilitated the change in belief, attitude, and practice. 

Therefore, the faculty became a living artifact in and 
as a result of the method. The process itself brought 
faculty, students, and administration together for the 
inquiry, and the scholarly and self -renewing attitudes of 
those participants changed into what seems to be an on-going 
and evolutionary process. This would support the idea that 
the process realxgned the departmental culture toward An 
emphasis on research, the pursuit of developing a scholar- 
practitioner profile, and a strong commitment to the new 
6®©rging culture. Transformation is with the process and not 
the product. At this stage of analysis, it is clear that the 
department is in a state of becoming, and not a state of 
being. 

Conclusion 

Using multicase methodology to examine the impact of 
ic changes resulted in a deeper understanding of 
the place of institutional research in institutional 
renewal. This evolutionary process of self-study, self- 
renewal, and mindful inquiry provided the opportunity for a 
faculty to begin engaging in critical self-examination. 
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kiiowledge brought forth from historical contexts, 
stakeholder voice, and commitment to the inquiry process 
resulted in both programmatic changes and an epistemological 
paradigm shift toward a scholar practitioner culture. 
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